Aaron Clark, 16 Aug 2023
Any opinions expressed in this presentation and on the following slides are solely those of the presenter and do not necessarily reflect those sponsoring the work
Quick Intro to {riskmetric} / {riskassessment}
Why create a shiny app?
Latest enhancements
Demo
Coming soon
Q&A
Consortia of ~50 companies (mostly pharma and biotech)
Focus on designing/building a framework to validate R and R packages with an eye to all open-source software.
The main goal is ensuring proper validation documentation exists in accordance with expectations of regulatory agencies.
Checkout pharmar.org to learn about all work streams and get involved!
is a framework to quantify an R package’s “risk” by assessing several meaningful metrics designed to evaluate package development best practices, code documentation, community engagement, and development sustainability.
is a full-fledged R package containing a shiny front-end that augments the utility of {riskmetric}. The application’s goal is to provide a central hub for an organization to review and assess the risk of R packages, providing handy tools and guide rails along the way.
Sometimes “quality” is measurable! Software dev best practices dictate an R-package should have:
18 total assessments (to date)!
Main goal: help those those making “package inclusion” requests for GxP environments. So, the app’s highest and best use is all about two things:
Main goal: help those those making “package inclusion” requests for GxP environments. They need support to…
{riskmetric}{riskmetric} on the same machine with the same environment – creating a central hub for reproducibilityMost notably:
'Report Builder' & 'Database View'admin users to edit roles and privileges on the fly
The feedback loop is crucial! Submit an issue on GitHub today.
'Report Builder' Face LiftA more holistic Report Builder! Allowing users to define what content shows up in the report.
Users can now compose a long form “Package Summary” to keep track of more pertinent items (perhaps non-{riskmetric} items) for a more rounded package review.
'Database View' Face LiftMore useful content to ingest!
Edit in-app or via the config file! What’s new?
Example config (YML) file:
default:
assessment_db: database.sqlite
loggit_json: loggit.json
credential_db: credentials.sqlite
credentials:
roles:
- admin
- lead
- reviewer
- viewer
privileges:
admin: [admin, weight_adjust, auto_decision_adjust, final_decision, revert_decision, add_package, delete_package, overall_comment, general_comment]
lead: [weight_adjust, auto_decision_adjust, final_decision, revert_decision, add_package, delete_package, overall_comment, general_comment]
reviewer: [add_package, general_comment]
viewer:
decisions:
categories:
- Low Risk
- Medium Risk
- High Risk
rules:
Medium Risk:
- .398
- .639
High Risk:
- .639
- 1
colors:
Medium Risk: !expr grDevices::rgb(52, 235, 229, maxColorValue = 255)
metric_weights:
covr_coverage: 0
has_vignettes: 2
Automated a risk-based “decision triage” based on pre-defined rules, saving time & effort
Decisions impacted upon pkg upload:
Manage who’s involved in the review process via user authentication & role management
Augment {riskmetric} evaluation with a more manual package review process by browsing source code
Most notably:
'Report Builder' & 'Database View'admin users to edit roles and privileges on the flyThe feedback loop is crucial! Submit an issue on GitHub today.
{prodlim} package
Demo on Shinyapps.io!
Test drive now: bit.ly/raa_demo
In it’s early stages, a new data package containing the {riskmetric} assessments & scores for all packages on CRAN.
A suite of hands-on review tools (from our friends at GSK) will allow users to browse unit tests, source code, and man (doc) pages for any exported function.
bit.ly/raa_demo
bit.ly/raa_gh
bit.ly/raa_survey
pharmar.org/